Columbia Kermit/kermit.zip/newsgroups/misc.20000114-20000217/[e-mail] _Thu Jan 27 22:58:17 2000.msg

Internet Message Format | 2020-01-01 | 10KB

Return-Path: <[e-mail]>
Received: from newsmaster.cc.columbia.edu (newsmaster.cc.columbia.edu [128.59.59.30])
    by watsun.cc.columbia.edu (8.8.5/8.8.5) with ESMTP id WAA02486
    for <[e-mail]>; Thu, 27 Jan 2000 22:58:17 -0500 (EST)
Received: (from news@localhost)
    by newsmaster.cc.columbia.edu (8.8.5/8.8.5) id WAA07282
    for [e-mail]; Thu, 27 Jan 2000 22:38:35 -0500 (EST)
X-Authentication-Warning: newsmaster.cc.columbia.edu: news set sender to <news> using -f
From: [e-mail] (Joe Doupnik)
Subject: Re: Case Study #10: Atomic File Movement
Message-ID: <nJLJ3X13aW$[e-mail]>
Date: 27 Jan 00 20:26:23 MDT
Organization: Utah State University
To: [e-mail]

In article <86qta3$2a0$[e-mail]>, [e-mail] (Frank da Cruz) writes:
> In article <gSF8krj$[e-mail]>, Joe Doupnik <[e-mail]> wrote:
> :     A "friendly amendment." While the Kermit protocol, and TCP, do an
> : acceptable job of confirming stages of work are completed, those techniques
> : do not remove ambiguity. Frank correctly states "somewhat immune." Old
> : packets whose sequence numbers have wrapped to the proper current value,
> : badly garbled ones with apparently legit contents (CRC checks are hardly
> : perfect), and packets delivered by mistake to the wrong session, are three
> : serious concerns for protocol designers because they confuse the normal
> : stage by stage confirmations. TCP uses three way handshakes, extra steps
> : to extend sequence numbers in some circumstances, and pseudo headers, to
> : help reduce false indications. Kermit does a pretty good job too, but not
> : to the extent that TCP goes.
> :     The two hill army problem remains when one gets serious about comms.
> : As stated, there is no certainty in the exchange, only approximation to it.
> :
> I meant to get back to this earlier, (so as not to leave an unsettling
> impression with readers who don't study these topics) but better late than
> never.
>
> I believe most of Joe's observations pertain more to TCP and IP than to
> Kermit:
>
>  . Old packets whose sequence numbers have wrapped.  This can happen in
>    TCP/IP because it's a worldwide packet-switched network.  A TCP packet
>    (encapsulated within an IP packet) can be stuck in the network for
>    minutes, hours, days, or weeks, and then show up after the sequence
>    number space has recycled one or more times, and then it can cause
>    trouble unless there is a higher (than TCP) level of checking.  But
>    Kermit connections are either point-to-point in fact, or in effect, so
>    packets don't lurk in odd crannies of the world and reappear at a later
>    time -- at least not late enough to cause confusion about packet
>    numbers.  Why?  Because (in the non-streaming case) every Kermit packet
>    must be acknowledged.  The window can't be larger than half the
>    sequence number space, and it can't advance until the oldest packet in
>    the window is acknowledged.  This technique, called "sliding windows
>    with selective retransmission", is more conservative and robust than
>    the technique TCP uses in preventing packet sequence number ambiguity.
>
>  . Packets are delivered by mistake to the wrong session.  Can't happen in
>    Kermit because there is only one session.
>
>  . Packets can be garbled to look like other packets.  Yes, this can
>    happen in any communictions protocol with some calculable probability.
>    But let's look at the consequence in the context of atomic file
>    movement.  First, it is possible (but highly unlikely) that a data
>    packet can be corrupted in such a way that its CRC will still be
>    correct, thus allowing bad data into a file (but only if the packet
>    sequence number, length, and other controls remain valid).  Of course
>    this can happen in any communications protocol; there is a whole
>    literature on the subject.  But what about the progress of the protocol
>    itself?  Each possible happenstance and its consequences can be
>    examined in turn.  For example, an ACK can turn into a NAK with the
>    same sequence number.  No harm is done.  A NAK can turn into an ACK for
>    the same packet.  Again, no harm is done (because the seemingly ACK'd
>    packet will missing at the receiver, and this will cause the transfer
>    to fail eventually.)  An ACK is turned into something besides an ACK or
>    NAK: then we have an illegal packet type and the transfer fails.  An
>    ACK is turned into an ACK with a different sequence number; if it's an
>    "old" sequence number it is ignored and no harm is done; if it's a
>    "new" one, the sender will catch the error ("You ACK'd a packet I didn't
>    send").  And so on.
>
> In other words, I think it is safe to say that the chances are practically
> negligible that a Kermit transfer will appear to succeed when it failed.
> Except perhaps for possible data corruption, which all protocols are subject
> to; as noted in the literature, the number of errors that a CRC will not
> catch is very small, and the probability that exactly such an error will
> occur, out of all the kinds of errors that can occur, is much smaller still.
>
> And in fact, in 20 years of experience with Kermit transfers, I can't recall
> a single confirmed report of the protocol reporting success when the
> transfer failed.
>
> - Frank
-------
    Faithful readers can see an academic discussion right away. What
Frank and I are doing here is exploring the outer limits of protocols
to see just how close one can come to meeting a requirement that both
file transmiter and receiver agree that the file has made it across intact.
It's an interesting puzzle, actually, because we learn neat things about
particular protocols.
    Take the stray ACK arriving out of blue at the worse possible
moment (masquerading as that very ACK from another session). The stray
can readily happen because the comms pathway is long, such as the
Internet with its many routers and paths. Ok, so it could happen, yet
it would be rare indeed to line up just so. What if it did, however.
An ACK tells the transmitter to dismiss the held packet and worry about
the next. The file isn't changed at this point. But the receiver has
its own ideas of propriety and needs the packet which is missing but is
covered by that stray ACK. So, the receiver, being a good protocol
engine, declines to let that go unnoticed and insists upon the missing
packet being filled in soon; it won't leave home without it. Whew!
    A more serious stray is a data packet arriving just before the
real one and the real one is rejected as a duplicate (same sequence
number etc). Here we get corruption and don't know it. Bad guys like
to play tricks like this with security related traffic. Such spoofing
can be taken to extremes of masquerading as the real file sender or
receiver, and so on.
    On getting sessions muddled. The way this can happen is to
run two Kermits over the same underlying transport protocol stack,
say TCP/IP but it could be others. A "small mistake, fixed in the next
release" could deliver a packet to the wrong Kermit. Nothing bad happens
unless sequence numbers and packet kind line up just right, but in
principle they might when the damage caused is greatest (someone or
other's principle). Things of this kind cause rotten comms and we
do something about it well before file transfers could make mistakes.
    By the way, if this happens with TCP/IP then a nasty message
is returned from that wrong stack and the affected session can be
abruptly terminated. Kermit is forgiving and keeps on running.
    Garbling packets to look like other pristine packets is much
more difficult indeed, an art form almost. But it could happen.

    Of these three situations the least likely to cause trouble
is the third, garbling to look nice. It's just too hard to get a good
match. But there is that opening where bytes get swapped wrong in a
buffer deep down in the hardware and by chance the CRC check is ok.
The first kind, stray packet, is more likely, but not frequent enough to
draw attention of diligent system managers. We would have to have
delays arranged just right for sequence numbers to have gone round one
whole cycle. That's a lot of storage in the net, and long term storage
there is very unlikely (it's tough getting short term storage, hence
packet loss under congestion). The most easily arranged boo-boo is that
mis-delivery with parallel Kermits, because it's a programming error
(the error would happen a lot) or a hardware glitch (much less likely).
Even then things would have to line up just so to make a difference.
    What I am saying is, guarantees can't be absolute. They are
normally very very good indeed, but not absolute. Making a typing
error denoting the file of interest is vastly more likely to occur,
not to mention all the errors going into making the file and reading
it later (given today's standards in programming, sigh).
    If you are wondering if the Kermit protocol is more immune to
these things than say TCP/IP. The answer is, I do believe, Kermit is
more robust than TCP/IP; it has slightly fewer windows of opportunity.
    Frank is saying the same things, but he is trying to be more
reassuring. Fine. What I am saying is absolute certainty is unobtainable,
no matter where one chooses to draw the line over which data flows.
Please do check your SCSI bus and RAID controller cache memory, and
type more carefully next time.
    Now wasn't that educational? I thought so; rather fun too.
    Joe D.